Deblocking filters are used in video coding standards in order to combat blocking artifacts. The blocking artifacts arise because the original video is split into blocks which are processed relatively independently. The blocking artifacts can arise due to different intra prediction of blocks, quantization effects and motion compensation. Two particular variants of deblocking are described below.
These descriptions assume a vertical block boundary or border. The process is also done in the same way for horizontal block boundaries.
H.264 Deblocking
In state of the art video coding such as H.264 there is an adaptive de-blocking filter/loop filter after prediction and residual reconstruction, but before storage of the reconstruction for later reference when encoding or decoding subsequent frames. The deblocking filtering consists of several steps such as filter decisions, filtering operations, a clipping function and changes of pixel values. The decision to filter the border or not is made based on evaluating several conditions. Filter decisions depend on macro block (MB) type, motion vector (MV) difference between neighboring blocks, whether neighboring blocks have coded residuals and on the local structure of the current and/or neighboring blocks.
The amount of filtering for a pixel depends on the position of that pixel relative to the block boundary and on the quantization parameter (QP) value used for residual coding. Here below a to h represent pixel values across a vertical block boundary.                a b c d|e f g h        
The filter decision is based on comparing three pixel differences with three thresholds. The thresholds are adapted to the QP.
If the following conditions are fulfilled the filtering is done:abs(d−e)<thr1,abs(c−d)<thr2, andabs(e−f)<thr2where thr1 and thr2 are functions of QP.
There are two filtering modes in H.264. In the first filtering mode (normal filtering), filtering can be described with a delta value that the filtering changes the current pixel value with. The filtering for the pixel closest to the block boundary is:d′=d+delta and e′=e−deltawhere delta has been clipped off to a threshold ±thr3 to a value that is constrained by the QP. d′ is here the pixel value at position d after filtering and e′ is the pixel value after filtering at position e. More filtering is allowed for high QP than for low QP.
Clipping can be described as:delta_clipped=max(−thr3,min(thr3,delta))where thr3 is controlling the filter strength. A larger value of thr3 means that the filtering is stronger, which in turns means that a stronger low-pass filtering effect will happen.
The filter strength can be increased if any of the following two conditions also holds:abs(b−d)<thr2 and abs(e−g)<thr2
The filter strength is adapted by clipping the delta less, e.g. to allow for more variation.
The second filtering mode (strong filtering) is applied for intra macroblock boundaries only, when the following condition is fulfilled:abs(d−e)<thr1/4.
The thresholds thr1, thr2 and thr3 are derived from table lookup using QP as index. Each slice can contain modifications of thr2 and thr3 using slice_beta_offset_div2 and thr1 using slice_alpha_c0_offset_div2. The slice parameters 2×slice_beta_offset_div2 and 2×slice_alpha_c0_offset_div2 are added to the current QP index before table lookup of thr2/thr3 and thr1 respectively.
Deblocking in HEVC Draft
Here below p0 to p3 and q0 to q3 represent pixel values across a vertical block boundary.                p3 p2 p1 p0|q0 q1 q2 q3         
In the draft HEVC specification, the deblocking filter works differently than H.264. The filtering is performed if at least one of the blocks on the side of the border is intra, or has non-zero coefficients, or the difference between the motion vector components of the blocks is greater than or equal to one integer pixel. For example, if one is filtering the border between the blocks A and B below, then the following condition should satisfy for the block boundary to be filtered:
ABp30p20p10p00q00q10q20q30p31p21p11p01q01q11q21q31p32p22p12p02q02q12q22q32p33p23p13p03q03q13q23q33dp0=|p20−2×p10+p00|dp3=|p23−2×p13+p03|dq0=|q20−2×q10+q00|dq3=|q23−2×q13+q03|dpq0=dp0+dq0dpq3=dp3+dq3dp=dp0+dp3dq=dq0+dq3  (1)
The variable d is derived as follows:d=dpq0+dpq3  (2)
The deblocking filtering is performed on the block boundary for lines i=0 . . . 3 if the following condition holds:d<β  (3)where β depends on the quantization parameter. In the draft HEVC specification, there is a table, see Table 1 below, for looking up the value of β using Q as the table index. β increases with increasing quantization parameter.
If the condition in equation (3) above is fulfilled and filtering is done between blocks A and B, one of two types of filtering (weak or strong filtering) is performed. The choice between the strong and the weak filtering is done separately for each line depending on the following conditions. For lines i=0, 3, strong filtering is performed if all the following conditions are true, otherwise, weak filtering is performed:2×dpqi<(β>>2)  (4)and (|p3i−p0i|+|q0i−q3i|)<(β>>3)  (5)and |p0i−q0i|<((5*tC+1)>>1),  (6)where tC and β depend on qP. Their relations are shown in Table 1 and further below.
TABLE 1Derivation of threshold variables β′ and tc′ from input QQ0123456789101112131415161718β′0000000000000000678tc′0000000000000000001Q19202122232425262728293031323334353637β′9101112131415161718202224262830323436tc′1111111122223333444Q383940414243444546474849505152535455β′384042444648505254565860626464646464tc′5566789910101111121213131414
The value of the variable β′ is determined as specified in Table 1 based on the parameter Q derived as:Q=Clip3(0,51,qP+(beta_offset_div2<<1))where qP is an average quantization parameter (QP) and beta_offset_div2 is the value of the variable beta_offset_div2 for the slice that contains the sample q00.
The variable β is derived as:β=β′×(1<<(BitDepth−8))
The value of the variable tc′ is determined as specified in Table 1 based on the parameter Q derived as:Q=Clip3(0,53,qP+2×(bS−1)+(tc_offset_div2<<1))where qP is an average quantization parameter (QP) and tc_offset_div2 is the value of the variable tc_offset_div2 for the slice that contains sample q00 Parameter bS is the boundary strength of the block boundary, which takes value 2 when one of the adjacent blocks is intra-predicted and value 1 in the other cases when the block boundary is processed by the deblocking filter (when bS is equal to 0, the block boundary is not processed by the deblocking filter).
The variable tc is derived as:tc=tc′×(1<<(BitDepth−8))
The parameters beta_offset_div2 and tc_offset_div2 are sent in the slice header or in a picture parameter set (PPS) as shown in the following.
Sequence Parameter Set RBSP Syntax
seq_parameter_set_rbsp( ) {Descriptor. . . bit_depth_luma_minus8ue(v) bit_depth_chroma_minus8ue(v). . .Picture Parameter Set RBSP Syntax
pic_parameter_set_rbsp( ) {Descriptor. . . pic_init_qp_minus26se(v) constrained_intra_pred_flagu(1) transform_skip_enabled_flagu(1) cu_qp_delta_enabled_flagu(1) if ( cu_qp_delta_enabled_flag )  diff_cu_qp_delta_depthue(v) pic_cb_qp_offsetse(v) pic_cr_qp_offsetse(v) pic_slice_level_chroma_qp_offsets_present_flagu(1)... deblocking_filter_control_present_flagu(1) if( deblocking_filter_control_present_flag ) {  deblocking_filter_override_enabled_flagu(1)  pps_disable_deblocking_filter_flagu(1)  if( !pps_disable_deblocking_filter_flag ) {   beta_offset_div2se(v)   tc_offset_div2se(v)  } }. . .General Slice Header Syntax
De-slice_header( ) {scriptor. . .  slice_qp_deltase(v)  if( pic_slice_level_chroma_qp_offsets_present_flag ) {   slice_cb_qp_offsetse(v)   slice_cr_qp_offsetse(v)  }  if( deblocking_filter_control_present_flag ) {   if( deblocking_filter_override_enabled_flag )    deblocking_filter_override_flagu(1)   if( deblocking_filter_override_flag ) {    slice_header_disable_deblocking_filter_flagu(1)    if( !slice_header_disable_deblocking_filter_flag ) {     beta_offset_div2se(v)     tc_offset_div2se(v)    }   }  }. . .
slice_qp_delta specifies the initial value of QP to be used for the coding blocks in the slice until modified by the value of CuQpDelta in the coding unit layer. The initial QP quantization parameter for the slice is computed as:SliceQP=26+pic_init_qp_minus26+slice_qp_delta
The value of slice_qp_delta shall be limited such that SliceQP is in the range of −QpBdOffset to +51, inclusive.
The two filtering modes (weak and strong filtering) in the HEVC draft look like in the following:
Weak Filtering
Weak filtering is performed based on the above conditions. The actual filtering works by computing an offset (Δ) for each of the lines i that the weak filter has been selected for. The following weak filtering procedure is applied for every line, where it has been chosen. In the following algorithm, the variables p0 . . . p2 and q0 . . . q2 are assigned the following values (from row/column i) where p0=p0i, p1=p1i, p2=p2i, q0=q0i, q1=q1i, q2=q2i. One can see that if the weak filtering is performed, one to two pixels are modified at each side of the block boundary:
Δ = ( 9×( q0 − p0 ) − 3×( q1 − p1) + 8 ) >> 4if( abs( Δ ) < 10×tc ){Δ = Clip3( −tc, tc, Δ )p0′ = Clip1Y( p0 + Δ )q0′ = Clip1Y( q0 − Δ )if( dp < ( β + ( β >> 1 ) ) >> 3 ){Δp = Clip3( −( tc >> 1 ), tc >> 1, ( ( ( p2 + p0 + 1 ) >> 1 ) −p1 + Δ ) >>1 )p1′ = Clip1Y( p1 + Δp)}if( dq < ( β + (β >> 1 ) ) >> 3){Δq = Clip3( −( tc >> 1 ), tc >> 1, ( ( ( q2 + q0 + 1 ) >> 1 ) −q1 − Δ ) >>1 )q1′ = Clip1Y( q1 + Δq )}}where Clip is defined asx′=Clip3(A,B,x), x′=x or if x<A then x′=A or if x>B then x′=B andClip1Y(x)=Clip3(0,(1<<BitDepthY)−1,x)where BitDepthY is the bit depth, for example 8 or 10.
Clipping operations are used in deblocking to avoid excessive filtering. The clipping parameter tC is derived from Table 1 and depends on the average of the quantization parameter QP of the two adjacent blocks which determines how coarse the quantization is. The clipping parameter tC determines what are the maximum allowed modifications to the sample value. Hence, larger values of tC have an effect of allowing stronger modifications to the sample values by the deblocking filtering, whereas a lower value of tC decreases the allowed modifications of samples by deblocking and therefore results in weaker filtering.
Strong Filtering
Strong filtering mode is performed for a line i of pixels by the following set of operations, where p0=p0i, p1=p1i, p2=p2i, q0=q0i, q1=q1i, q2=q2i:p0′=Clip3(p0−2×tc,p0+2×tc,(p2+2×p1+2×p0+2×q0+q1+4)>>3)p1′=Clip3(p1−2×tc,p1+2×tc,(p2+p1+p0+q0+2)>>2)p2′=Clip3(p2−2×tc,p2+2×tc,(2×p3+3×p2+p1+p0+q0+4)>>3)q0′=Clip3(q0−2×tc,q0+2×tc,(p1+2×p0+2×q0+2×q1+q2+4)>>3)q1′=Clip3(q1−2×tc,q1+2×tc,(p0+q0+q1+q2+2)>>2)q2′=Clip3(q2−2×tc,q2+2×tc,(p0+q0+q1+3×q2+2×q3+4)>>3)
The quantization parameter (QP) controls the bitrate and the fidelity of the video signal. In a hybrid codec, the QP is used for quantizing and de-quantizing transform coefficients that contain residual data after prediction. A low QP value will result in low coding distortion but will also result in a lot of coefficients that will cost a lot of bits. A high QP results in less coefficients, which will result in high coding distortion but will not cost many bits to code.
The slice QP specifies the initial value of the QP to be used for the macroblocks/coding units (CUs) in the slice. The slice QP is used for the first macroblock/CU in the slice. Then delta QP syntax elements are sent between each macroblock/CU to change the QP value. Typically, a delta QP is only sent for macroblocks/CUs that contain transform coefficients. No delta QP is sent for blocks that do not contain coefficients since the QP value then has no impact on the pixel reconstruction process.
It may happen that all delta QP values are zero. In this case, the slice QP is used for all blocks in the entire slice.
It is well known that subjective video compression gains can be achieved by using what is called adaptive QP where the quantization parameter (QP) is changed within pictures. Applying a low QP in an area will generate more bits compared to a higher QP but the coding distortion will be lower. A lower QP can be used on areas that have smooth textures and a higher QP can be used where the spatial activity is higher. This is a good idea since the human visual system will easily detect distortion in a smooth area, while the same amount of distortion in a highly textured area will go unnoticed. However, the range of the QP adaptation in the slice is limited and depends on the initial value of the slice/picture QP.
In video coding, hierarchical coding structures are often used. The hierarchical QP coding often uses higher values of QP for coding of pictures/slices, which are higher in the coding hierarchy, and might be used together with or without decoding picture order rearrangement. Hierarchical QP coding, i.e. QP toggling, usually results in higher objective and subjective quality of the coded video. The adjustment of quality can be done by using higher value of initial QP delta as well as using adjustment to the Lagrange multiplier λ used in the rate-distortion optimization.
One of the problems with hierarchical coding structures is that the pictures/slices, which lay higher up in the hierarchy, are coded with lower quality. Since these pictures are often coded with higher values of initial picture QP and/or slice QP, the deblocking thresholds automatically adapt to this coding structure since the higher value of QP is used with higher depth in the coding hierarchy and thresholds are QP dependent. However, since the quality of pictures in the higher layers of hierarchy is decreased, some blocking artifacts can still be visible, especially on the large transform size grid. Moreover, using higher values of Lagrange multiplier λ in the rate-distortion optimization when coding pictures with greater depth in the coding hierarchy can further reduce the quality of these pictures and makes it more difficult for deblocking to attenuate the blocking artifacts.